Population-based surveillance for birth defects potentially related to Zika virus infection including 3-year mortality and developmental outcomes, and Early Intervention Program service use-New York City, 2016 birth cohort.

BACKGROUND: In response to the 2015-2017 Zika virus outbreak, New York City (NYC) identified and monitored infants with birth defects potentially related to congenital Zika virus. METHODS: Administrative data matches were used to describe the birth characteristics of children born in 2016 meeting screening criteria for birth defects potentially related to congenital Zika virus infection relative to other NYC births and to monitor mortality and Early Intervention Program use through age 2. RESULTS: Among 120,367 children born in NYC in 2016, 463 met screening criteria and 155 met the Centers for Disease Control and Prevention's case definition for birth defects potentially related to congenital Zika virus infection (1.3 per 1000; 95% confidence interval [CI], 1.1-1.5). Post-neonatal deaths occurred among 7.7% of cases (12) and 5.2% of non-cases (8). Odds of referral to the Early intervention Program among children who met screening criteria were lower among children of mothers who were married (OR, 0.60; 95% CI, 0.37-0.97) and among children not classified as cases whose mothers were born in Latin America and the Caribbean (OR, 0.59; 95% CI, 0.37-1.09). DISCUSSION: Prevalence of birth defects potentially related to congenital Zika virus infection was similar to that seen in other jurisdictions without local transmission. Birth defects attributable to congenital Zika virus infection may also have been present among screened children who did not meet the case definition.

Continuity of Early Intervention Services in New York City During the COVID-19 Pandemic.

In response to COVID-19, the New York City Early Intervention (EI) Program rapidly transitioned from in-person to teletherapy services. We describe the timing of service resumption among children who received EI services between March 1 and March 17, 2020. The proportion of children who transitioned to teletherapy-only was 25% as of March 24, rising to 78% by July 6. By December 31, 2020, 87% of the cohort had resumed either teletherapy or in-person services. Child age, race, language, and neighborhood poverty all predicted service resumption timing. Children with a diagnosis of autism spectrum disorder were more likely to transition to teletherapy, and children with only 1-2 domains of delay were more likely to discontinue services altogether. Continuity of EI services during the COVID-19 public health emergency was a critical priority. Timely policy changes facilitated swift return to services and avoided exacerbation of the long-standing racial disparities in access to EI services.

Early intervention and special education in New York City: Patterns of service use and disparities affecting children of colour.

BACKGROUND: Compliance with the requirements of the Individuals with Disabilities Education Act (IDEA) in the United States is monitored through review of cross-sectional reports from three discrete, age-defined programmes (early intervention [EI], early childhood special education [ECSE)] and school-age special education [SE]) to promote the timely, efficient and effective delivery of appropriate services to all eligible children. Analysis of longitudinal data is required to discern how children use services across programmes to provide the necessary context for IDEA oversight and to identify areas for programme or policy interventions to reduce barriers to service use and promote equity. METHODS: We applied sequence analysis to a data linkage across five public record systems among 15 626 New York City children born in 1998 who had records from birth through third grade. RESULTS: Five predominant patterns of service use were identified: (1) multiple therapies across EI/ECSE/SE (13%), (2) EI without transition to Department of Education schools or services (24%), (3) EI and intermittent ECSE/SE (16%), (4) older entry into EI and both speech and occupational therapy throughout ECSE/SE (9%) and (5) limited EI use and mostly speech therapy in ECSE/SE (38%). Each pattern had distinct demographics (e.g., pattern 2 was disproportionately White and from low poverty neighbourhoods; pattern 4 was disproportionately male and Black; pattern 5 was disproportionately Latino) and academic outcomes (e.g., pattern 1 had largest proportion in a SE school and not tested in third grade; pattern 3 had third grade tests scores that were similar to overall citywide mean scores). CONCLUSIONS: The differences in demographic profiles across the five patterns of service use illustrate the systemic inequities in the delivery of these important services. Delayed entry and limited use of EI services among children of colour underscore the need for equity goals to increase early referral and optimize service use.

Air pollution, children's academic achievement and the potential mediating role of preterm birth.

INTRODUCTION: Previous research has observed relationships between higher prenatal exposure to air pollutants and neurodevelopmental and academic outcomes later in childhood. Identifying intermediate outcomes mediating this relationship would inform prevention and intervention efforts. We aimed to investigate if previously observed associations between prenatal exposure to common urban air pollutants, diesel and perchloroethylene, and performance on third grade standardized tests were mediated through increased risk of preterm birth. METHODS: Data from the 1994-1998 birth cohorts within the New York City Longitudinal Study of Early Development were included in this analysis. Exposure was determined by linking the mother's residence at the time of delivery to the U.S. EPA's 1996 National Air Toxic Assessment of estimated ambient concentrations of diesel and perchloroethylene. Children's third grade standardized math and language tests were used as the markers for academic achievement. Missing data on covariates were imputed, while participants with missing information on gestational age and test scores were excluded. Linear regression models and causal mediation analysis were used to examine potential mediation by preterm birth. RESULTS: In total, 187,723 and 196,122 participants were included in language and math analyses, respectively. Children with exposure to the fourth quartile of diesel or perchloroethylene had approximately 0.03 (95%CI: 0.02, 0.04) lower math z-scores when compared to individuals with exposure in the first quartile, although there was no consistent decreasing trend in math z-scores over increasing quartiles of diesel or perchloroethylene. We did not find evidence of mediation by preterm birth or exposure-mediator interaction in our models. CONCLUSION: We did not find evidence that observed relationships between exposure to common urban air pollutants and test z-scores in childhood were mediated through an increased risk of preterm birth. This suggests other pathways between early exposure to air pollution and neurodevelopment should be investigated with causal mediation approaches.

Receipt of Early Intervention Services Before Age 3 Years and Performance on Third-Grade Standardized Tests Among Children Exposed to Lead.

IMPORTANCE: Research has shown that early intervention programs can improve academic outcomes of children with developmental delays. It has been suggested that similar programs may combat the deleterious effects of lead on children's neurodevelopment. However, to our knowledge, there are no published studies examining this possibility. OBJECTIVE: The objective of this study was to estimate the association between receipt of early intervention services and third-grade standardized test scores among children exposed to lead before age 3 years. DESIGN, SETTING, AND PARTICIPANTS: Cohort study including children born in New York City, New York, from 1994 to 1998 within an administrative data linkage of birth, lead monitoring, early intervention, and education data systems. Participants had a blood lead level of 4 µg/dL or greater at any point before age 3 years and later attended public school in New York City. EXPOSURES: Any use of early intervention services from birth through age 3 years. MAIN OUTCOMES AND MEASURES: Children who did or did not receive early intervention services were matched using propensity scores. Linear and log-binomial regression were used to estimate the association between receipt of early intervention services before age 3 years and standardized test scores in math and English-language arts in third grade. RESULTS: There were 2986 children exposed to lead who received early intervention services before age 36 months. Of these children, 2757 were propensity score-matched to 8160 children who did not receive services. Children who received early intervention services did 7% (95% CI, 3%-12%) of an SD better on math and 10% (95% CI, 5%-14%) of an SD better on English-language arts tests than children who did not receive services. In addition, children who received services were 14% (95% CI, 9%-19%) and 16% (95% CI, 9%-23%) more likely to meet test-based standards in math and English-language arts, respectively, than children who did not receive services. These associations became larger in magnitude when analyses were restricted to children with higher blood lead levels. CONCLUSIONS AND RELEVANCE: By leveraging existing public health data, this study found evidence that receipt of early intervention services may benefit the academic performance of children exposed to lead early in life.

Evaluation of a health information exchange system for microcephaly case-finding - New York City, 2013-2015.

BACKGROUND: Birth defects surveillance in the United States is conducted principally by review of routine but lagged reporting to statewide congenital malformations registries of diagnoses by hospitals or other health care providers, a process that is not designed to rapidly detect changes in prevalence. Health information exchange (HIE) systems are well suited for rapid surveillance, but information is limited about their effectiveness at detecting birth defects. We evaluated HIE data to detect microcephaly diagnosed at birth during January 1, 2013-December 31, 2015 before known introduction of Zika virus in North America. METHODS: Data from an HIE system were queried for microcephaly diagnostic codes on day of birth or during the first two days after birth at three Bronx hospitals for births to New York City resident mothers. Suspected cases identified by HIE data were compared with microcephaly cases that had been identified through direct inquiry of hospital records and confirmed by chart abstraction in a previous study of the same cohort. RESULTS: Of 16,910 live births, 43 suspected microcephaly cases were identified through an HIE system compared to 67 confirmed cases that had been identified as part of the prior study. A total of 39 confirmed cases were found by both studies (sensitivity = 58.21%, 95% CI: 45.52-70.15%; positive predictive value = 90.70%, 95% CI: 77.86-97.41%; negative predictive value = 99.83%, 95% CI: 99.76-99.89% for HIE data). CONCLUSION: Despite limitations, HIE systems could be used for rapid newborn microcephaly surveillance, especially in the many jurisdictions where more labor-intensive approaches are not feasible. Future work is needed to improve electronic medical record documentation quality to improve sensitivity and reduce misclassification.

Large for Gestational Age and Risk for Academic Delays and Learning Disabilities: Assessing Modification by Maternal Obesity and Diabetes.

The objective of this study was to examine academic delays for children born large for gestational age (LGA) and assess effect modification by maternal obesity and diabetes and then to characterize risks for LGA for those with a mediating condition. Cohort data were obtained from the New York City Longitudinal Study of Early Development, linking birth and educational records (n = 125,542). Logistic regression was used to compare children born LGA (>90th percentile) to those born appropriate weight (5-89th percentile) for risk of not meeting proficiency on assessments in the third grade and being referred to special education. Among children of women with gestational diabetes, children born LGA had an increased risk of underperforming in mathematics (ARR: 1.18 (95% CI: 1.07-1.31)) and for being referred for special education (ARR: 1.18 (95% CI: 1.02-1.37)). Children born LGA but of women who did not have gestational diabetes had a slightly decreased risk of academic underperformance (mathematics-ARR: 0.94 (95% CI: 0.90-0.97); Language arts-ARR: 0.96 (95% CI: 0.94-0.99)). Children born to women with gestational diabetes with an inadequate number of prenatal care visits were at increased risk of being born LGA, compared to those receiving extensive care (ARR: 1.67 (95% CI: 1.20-2.33)). Children born LGA of women with diabetes were at increased risk of delays; greater utilization of prenatal care among these diabetic women may decrease the incidence of LGA births.

Birth characteristics of children who used early intervention and special education services in New York City.

BACKGROUND: Early intervention (EI) and special education (SE) are beneficial for children with developmental disabilities and/or delays and their families, yet there are disparities in service use. We sought to identify the birth characteristics that predict EI/SE service use patterns. METHODS: We conducted a retrospective cohort study using linked administrative data from five sources for all children born in 1998 to New York City resident mothers. Multinomial regression was used to identify birth characteristics that predicted predominant patterns of service use. RESULTS: Children with service use patterns characterized by late or limited/no EI use were more likely to be first-born children and have Black or Latina mothers. Children born with a gestational age ≤31 weeks were more likely to enter services early. Early term gestational age was associated with patterns of service use common to children with pervasive developmental delay, and maternal obesity was associated with the initiation of speech therapy at the time of entry into school. CONCLUSIONS: Maternal racial disparities existed for patterns of EI/SE service use. Specific birth characteristics, such as parity and gestational age, may be useful to better identify children who are at risk for suboptimal EI use.

Characteristics of Mothers and Infants Living in Homeless Shelters and Public Housing in New York City.

Introduction Homelessness can result in poor health. The number of families with children living in NYC homeless shelters increased 55% from 2008 to 2014. Half of children living in shelter in 2014 were younger than 6 years old. We compared demographics and health outcomes of mothers and infants residing in NYC homeless shelters to those residing in public housing in this cross-sectional study. Methods Addresses of NYC Department of Homeless Services shelters and NYC Housing Authority (NYCHA) developments were matched to NYC Department of Health birth certificate data for the years 2008-2013. Sociodemographic and health characteristics of newborns residing in shelters were compared to newborns in NYCHA housing using Chi square tests. Results Mothers residing in shelters were younger, more likely to be black and less likely to be Hispanic, more likely to have been born outside NYC and reside in the Bronx. Babies born to mothers living in shelter were more likely to have low birth weight (< 2500 g), be born preterm (< 37 gestational weeks), require assisted ventilation immediately following delivery, have a NICU admission, and use Medicaid. They were less likely to breastfeed within 5 days of delivery and be discharged to their residence. Discussion Homeless mothers and infants had poorer health outcomes compared with those living in public housing. Understanding the health disparities of homeless infants can provide guidance for developing future policies and research initiatives, which may be used to inform the development of new policies to improve health outcomes of homeless infants and their mothers.

Using Calibration to Reduce Measurement Error in Prevalence Estimates Based on Electronic Health Records.

INTRODUCTION: Increasing adoption of electronic health record (EHR) systems by health care providers presents an opportunity for EHR-based population health surveillance. EHR data, however, may be subject to measurement error because of factors such as data entry errors and lack of documentation by physicians. We investigated the use of a calibration model to reduce bias of prevalence estimates from the New York City (NYC) Macroscope, an EHR-based surveillance system. METHODS: We calibrated 6 health indicators to the 2013-2014 NYC Health and Nutrition Examination Survey (NYC HANES) data: hypertension, diabetes, smoking, obesity, influenza vaccination, and depression. We classified indicators into having low measurement error or high measurement error on the basis of whether the proportion of misclassification (ie, false-negative or false-positive cases) was greater than 15% in 190 reviewed charts. We compared bias (ie, absolute difference between NYC Macroscope estimates and NYC HANES estimates) before and after calibration. RESULTS: The health indicators with low measurement error had the same bias after calibration as before calibration (diabetes, 2.5 percentage points; smoking, 2.5 percentage points; obesity, 3.5 percentage points; hypertension, 1.1 percentage points). For indicators with high measurement error, bias decreased from 10.8 to 2.5 percentage points for depression, and from 26.7 to 8.4 percentage points for influenza vaccination. CONCLUSION: The calibration model has the potential to reduce bias of prevalence estimates from EHR-based surveillance systems for indicators with high measurement errors. Further research is warranted to assess the utility of the current calibration model for other EHR data and additional indicators.

Electronic health record case studies to advance environmental public health tracking.

Data from traditional public health surveillance systems can have some limitations, e.g., timeliness, geographic level, and amount of data accessible. Electronic health records (EHRs) could present an opportunity to supplement current sources of routinely collected surveillance data. The National Environmental Public Health Tracking Program (Tracking Program) sought to explore the use of EHRs for advancing environmental public health surveillance practices. The Tracking Program funded four state/local health departments to obtain and pilot the use of EHR data to address several issues including the challenges and technical requirements for accessing EHR data, and the core data elements required to integrate EHR data within their departments' Tracking Programs. The results of these pilot projects highlighted the potential of EHR data for public health surveillance of rare diseases that may lack comprehensive registries, and surveillance of prevalent health conditions or risk factors for health outcomes at a finer geographic level. EHRs therefore, may have potential to supplement traditional sources of public health surveillance data.

Comparing Prevalence Estimates From Population-Based Surveys to Inform Surveillance Using Electronic Health Records.

INTRODUCTION: Electronic health record (EHR) systems provide an opportunity to use a novel data source for population health surveillance. Validation studies that compare prevalence estimates from EHRs and surveys most often use difference testing, which can, because of large sample sizes, lead to detection of significant differences that are not meaningful. We explored a novel application of the two one-sided t test (TOST) to assess the equivalence of prevalence estimates in 2 population-based surveys to inform margin selection for validating EHR-based surveillance prevalence estimates derived from large samples. METHODS: We compared prevalence estimates of health indicators in the 2013 Community Health Survey (CHS) and the 2013-2014 New York City Health and Nutrition Examination Survey (NYC HANES) by using TOST, a 2-tailed t test, and other goodness-of-fit measures. RESULTS: A ±5 percentage-point equivalence margin for a TOST performed well for most health indicators. For health indicators with a prevalence estimate of less than 10% (extreme obesity [CHS, 3.5%; NYC HANES, 5.1%] and serious psychological distress [CHS, 5.2%; NYC HANES, 4.8%]), a ±2.5 percentage-point margin was more consistent with other goodness-of-fit measures than the larger percentage-point margins. CONCLUSION: A TOST with a ±5 percentage-point margin was useful in establishing equivalence, but a ±2.5 percentage-point margin may be appropriate for health indicators with a prevalence estimate of less than 10%. Equivalence testing can guide future efforts to validate EHR data.

Innovations in Population Health Surveillance: Using Electronic Health Records for Chronic Disease Surveillance.

With 87% of providers using electronic health records (EHRs) in the United States, EHRs have the potential to contribute to population health surveillance efforts. However, little is known about using EHR data outside syndromic surveillance and quality improvement. We created an EHR-based population health surveillance system called the New York City (NYC) Macroscope and assessed the validity of diabetes, hyperlipidemia, hypertension, smoking, obesity, depression, and influenza vaccination indicators. The NYC Macroscope uses aggregate data from a network of outpatient practices. We compared 2013 NYC Macroscope prevalence estimates with those from a population-based, in-person examination survey, the 2013-2014 NYC Health and Nutrition Examination Survey. NYC Macroscope diabetes, hypertension, smoking, and obesity prevalence indicators performed well, but depression and influenza vaccination estimates were substantially lower than were survey estimates. Ongoing validation will be important to monitor changes in validity over time as EHR networks mature and to assess new indicators. We discuss NYC's experience and how this project fits into the national context. Sharing lessons learned can help achieve the full potential of EHRs for population health surveillance.

Early-life exposure to air pollution and greater use of academic support services in childhood: a population-based cohort study of urban children.

BACKGROUND: There is a growing literature showing associations between prenatal and early-life exposure to air pollution and children's neurodevelopment. However, it is unclear if decrements in neurodevelopment observed in epidemiologic research translate into observable functional outcomes in the broader pediatric population. The objective of this study was to examine the association between early-life exposures to common urban air toxics and the use of academic support services, such as early intervention and special education within a population-based cohort of urban children. METHODS: Data for 201,559 children born between 1994 and 1998 in New York City were obtained through administrative data linkages between birth, early intervention and educational records. Use of academic support services was ascertained from birth through attendance in 3rd grade. Census tract at birth was used to assign estimates of annual average ambient concentrations of benzene, toluene, ethylbenzene and xylenes (BTEX) using the 1996 National Air Toxics Assessment. Discrete-time hazard models were fit to the data and adjusted for confounders including maternal, childhood and neighborhood factors. RESULTS: Children with higher exposures to BTEX compounds were more likely to receive academic support services later in childhood. For example, the adjusted hazard ratio comparing children exposed to the highest decile of benzene to those with lower exposure was 1.09 (95% confidence interval 1.05, 1.13). Results were consistent across individual BTEX compounds, for exposure metrics which summarized exposure to all four BTEX pollutants and for multiple sensitivity analyses. CONCLUSIONS: These findings suggest urban air pollution may affect children's neurodevelopment and educational trajectories. They also demonstrate the use of public health data systems to advance children's environmental health research.

Generalizability of Indicators from the New York City Macroscope Electronic Health Record Surveillance System to Systems Based on Other EHR Platforms.

INTRODUCTION: The New York City (NYC) Macroscope is an electronic health record (EHR) surveillance system based on a distributed network of primary care records from the Hub Population Health System. In a previous 3-part series published in eGEMS, we reported the validity of health indicators from the NYC Macroscope; however, questions remained regarding their generalizability to other EHR surveillance systems. METHODS: We abstracted primary care chart data from more than 20 EHR software systems for 142 participants of the 2013-14 NYC Health and Nutrition Examination Survey who did not contribute data to the NYC Macroscope. We then computed the sensitivity and specificity for indicators, comparing data abstracted from EHRs with survey data. RESULTS: Obesity and diabetes indicators had moderate to high sensitivity (0.81-0.96) and high specificity (0.94-0.98). Smoking status and hypertension indicators had moderate sensitivity (0.78-0.90) and moderate to high specificity (0.88-0.98); sensitivity improved when the sample was restricted to records from providers who attested to Stage 1 Meaningful Use. Hyperlipidemia indicators had moderate sensitivity (≥0.72) and low specificity (≤0.59), with minimal changes when restricting to Stage 1 Meaningful Use. DISCUSSION: Indicators for obesity and diabetes used in the NYC Macroscope can be adapted to other EHR surveillance systems with minimal validation. However, additional validation of smoking status and hypertension indicators is recommended and further development of hyperlipidemia indicators is needed. CONCLUSION: Our findings suggest that many of the EHR-based surveillance indicators developed and validated for the NYC Macroscope are generalizable for use in other EHR surveillance systems.

Characterizing Adults Receiving Primary Medical Care in New York City: Implications for Using Electronic Health Records for Chronic Disease Surveillance.

INTRODUCTION: Electronic health records (EHRs) from primary care providers can be used for chronic disease surveillance; however, EHR-based prevalence estimates may be biased toward people who seek care. This study sought to describe the characteristics of an in-care population and compare them with those of a not-in-care population to inform interpretation of EHR data. METHODS: We used data from the 2013-2014 New York City Health and Nutrition Examination Survey (NYC HANES), considered the gold standard for estimating disease prevalence, and the 2013 Community Health Survey, and classified participants as in care or not in care, on the basis of their report of seeing a health care provider in the previous year. We used χ(2) tests to compare the distribution of demographic characteristics, health care coverage and access, and chronic conditions between the 2 populations. RESULTS: According to the Community Health Survey, approximately 4.1 million (71.7%) adults aged 20 or older had seen a health care provider in the previous year; according to NYC HANES, approximately 4.7 million (75.1%) had. In both surveys, the in-care population was more likely to be older, female, non-Hispanic, and insured compared with the not-in-care population. The in-care population from the NYC HANES also had a higher prevalence of diabetes (16.7% vs 6.9%; P < .001), hypercholesterolemia (35.7% vs 22.3%; P < .001), and hypertension (35.5% vs 26.4%; P < .001) than the not-in-care population. CONCLUSION: Systematic differences between in-care and not-in-care populations warrant caution in using primary care data to generalize to the population at large. Future efforts to use primary care data for chronic disease surveillance need to consider the intended purpose of data collected in these systems as well as the characteristics of the population using primary care.

Association between prenatal exposure to ambient diesel particulate matter and perchloroethylene with children's 3rd grade standardized test scores.

UNLABELLED: The objective of this research was to determine if prenatal exposure to two common urban air pollutants, diesel and perchloroethylene, affects children's 3rd grade standardized test scores in mathematics and English language arts (ELA). Exposure estimates consisted of annual average ambient concentrations of diesel particulate matter and perchloroethylene obtained from the Environmental Protection Agency's 1996 National Air Toxics Assessment for the residential census tract at birth. Outcome data consisted of linked birth and educational records for 201,559 singleton, non-anomalous children born between 1994 and 1998 who attended New York City public schools. Quantile regression models were used to estimate the effects of these exposures on multiple points within the continuous distribution of standardized test scores. Modified Poisson regression models were used to calculate risk ratios (RR) and 95% confidence intervals (CI) of failing to meet curricula standards, an indicator derived from test scores. Models were adjusted for a number of maternal, neighborhood and childhood factors. Results showed that math scores were approximately 6% of a standard deviation lower for children exposed to the highest levels of both pollutants as compared to children with low levels of both pollutants. Children exposed to high levels of both pollutants also had the largest risk of failing to meet math test standards when compared to children with low levels of exposure to the pollutants (RR 1.10 95%CI 1.07,1.12 RR high perchloroethylene only 1.03 95%CI 1.00,1.06; RR high diesel PM only 1.02 95%CI 0.99,1.06). There was no association observed between exposure to the pollutants and failing to meet ELA standards. This study provides preliminary evidence of associations between prenatal exposure to urban air pollutants and lower academic outcomes. Additionally, these findings suggest that individual pollutants may additively impact health and point to the need to study the collective effects of air pollutant mixtures. KEY WORDS: air toxics, academic outcomes, urban health, tetrachloroethylene, air pollutant mixtures.

Design of the New York City Macroscope: Innovations in Population Health Surveillance Using Electronic Health Records.

INTRODUCTION: Electronic health records (EHRs) have the potential to offer real-time, inexpensive standardized health data about chronic health conditions. Despite rapid expansion, EHR data evaluations for chronic disease surveillance have been limited. We present design and methods for the New York City (NYC) Macroscope, an EHR-based chronic disease surveillance system. This methods report is the first in a three part series describing the development and validation of the NYC Macroscope. This report describes in detail the infrastructure underlying the NYC Macroscope; indicator definitions; design decisions that were made to maximize data quality; characteristics of the population sampled; completeness of data collected; and lessons learned from doing this work. The second report describes the methods used to evaluate the validity and robustness of NYC Macroscope prevalence estimates; presents validation results for estimates of obesity, smoking, depression and influenza vaccination; and discusses the implications of our findings for NYC and for other jurisdictions embarking on similar work. The third report applies the same validation methods to metabolic outcomes, including the prevalence, treatment and control of diabetes, hypertension and hyperlipidemia. METHODS: We designed the NYC Macroscope for comparison to a local "gold standard," the 2013-14 NYC Health and Nutrition Examination Survey, and the telephonic 2013 Community Health Survey. NYC Macroscope indicators covered prevalence, treatment, and control of diabetes, hypertension, and hyperlipidemia; and prevalence of influenza vaccination, obesity, depression and smoking. Indicators were stratified by age, sex, and neighborhood poverty, and weighted to the in-care NYC population and limited to primary care patients. Indicator queries were distributed to a virtual network of primary care practices; 392 practices and 716,076 adult patients were retained in the final sample. FINDINGS: The NYC Macroscope covered 10% of primary care providers and 15% of all adult patients in NYC in 2013 (8-47% of patients by neighborhood). Data completeness varied by domain from 98% for blood pressure among patients with hypertension to 33% for depression screening. DISCUSSION: Design and validation efforts undertaken by NYC are described here to provide one potential blueprint for leveraging EHRs for population health monitoring. To replicate a model like NYC Macroscope, jurisdictions should establish buy-in; build informatics capacity; use standard, simple case defnitions; establish documentation quality thresholds; restrict to primary care providers; and weight the sample to a target population.

Monitoring Prevalence, Treatment, and Control of Metabolic Conditions in New York City Adults Using 2013 Primary Care Electronic Health Records: A Surveillance Validation Study.

INTRODUCTION: Electronic health records (EHRs) can potentially extend chronic disease surveillance, but few EHR-based initiatives tracking population-based metrics have been validated for accuracy. We designed a new EHR-based population health surveillance system for New York City (NYC) known as NYC Macroscope. This report is the third in a 3-part series describing the development and validation of that system. The first report describes governance and technical infrastructure underlying the NYC Macroscope. The second report describes validation methods and presents validation results for estimates of obesity, smoking, depression and influenza vaccination. In this third paper we present validation findings for metabolic indicators (hypertension, hyperlipidemia, diabetes). METHODS: We compared EHR-based estimates to those from a gold standard surveillance source - the 2013-2014 NYC Health and Nutrition Examination Survey (NYC HANES) - overall and stratified by sex and age group, using the two one-sided test of equivalence and other validation criteria. RESULTS: EHR-based hypertension prevalence estimates were highly concordant with NYC HANES estimates. Diabetes prevalence estimates were highly concordant when measuring diagnosed diabetes but less so when incorporating laboratory results. Hypercholesterolemia prevalence estimates were less concordant overall. Measures to assess treatment and control of the 3 metabolic conditions performed poorly. DISCUSSION: While indicator performance was variable, findings here confirm that a carefully constructed EHR-based surveillance system can generate prevalence estimates comparable to those from gold-standard examination surveys for certain metabolic conditions such as hypertension and diabetes. CONCLUSIONS: Standardized EHR metrics have potential utility for surveillance at lower annual costs than surveys, especially as representativeness of contributing clinical practices to EHR-based surveillance systems increases.

Can Electronic Health Records Be Used for Population Health Surveillance? Validating Population Health Metrics Against Established Survey Data.

INTRODUCTION: Electronic health records (EHRs) offer potential for population health surveillance but EHR-based surveillance measures require validation prior to use. We assessed the validity of obesity, smoking, depression, and influenza vaccination indicators from a new EHR surveillance system, the New York City (NYC) Macroscope. This report is the second in a 3-part series describing the development and validation of the NYC Macroscope. The first report describes in detail the infrastructure underlying the NYC Macroscope; design decisions that were made to maximize data quality; characteristics of the population sampled; completeness of data collected; and lessons learned from doing this work. This second report, which addresses concerns related to sampling bias and data quality, describes the methods used to evaluate the validity and robustness of NYC Macroscope prevalence estimates; presents validation results for estimates of obesity, smoking, depression and influenza vaccination; and discusses the implications of our findings for NYC and for other jurisdictions embarking on similar work. The third report applies the same validation methods described in this report to metabolic outcomes, including the prevalence, treatment and control of diabetes, hypertension and hyperlipidemia. METHODS: NYC Macroscope prevalence estimates, overall and stratified by sex and age group, were compared to reference survey estimates for adult New Yorkers who reported visiting a doctor in the past year. Agreement was evaluated against 5 a priori criteria. Sensitivity and specificity were assessed by examining individual EHR records in a subsample of 48 survey participants. RESULTS: Among adult New Yorkers in care, the NYC Macroscope prevalence estimate for smoking (15.2%) fell between estimates from NYC HANES (17.7 %) and CHS (14.9%) and met all 5 a priori criteria. The NYC Macroscope obesity prevalence estimate (27.8%) also fell between the NYC HANES (31.3%) and CHS (24.7%) estimates, but met only 3 a priori criteria. Sensitivity and specificity exceeded 0.90 for both the smoking and obesity indicators. The NYC Macroscope estimates of depression and influenza vaccination prevalence were more than 10 percentage points lower than the estimates from either reference survey. While specificity was > 0.90 for both of these indicators, sensitivity was < 0.70. DISCUSSION: Through this work we have demonstrated that EHR data from a convenience sample of providers can produce acceptable estimates of smoking and obesity prevalence among adult New Yorkers in care; gained a better understanding of the challenges involved in estimating depression prevalence from EHRs; and identified areas for additional research regarding estimation of influenza vaccination prevalence. We have also shared lessons learned about how EHR indicators should be constructed and offer methodologic suggestions for validating them. CONCLUSIONS: This work adds to a rapidly emerging body of literature about how to define, collect and interpret EHR-based surveillance measures and may help guide other jurisdictions.